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Abstract 

We prove a maximum principle of optimal control of stochastic de- 
lay equations on infinite horizon. We establish first and second sufficient 
stochastic maximum principles as well as necessary conditions for that 
problem. We illustrate our results by an application to the optimal con- 
sumption rate from an economic quantity. 



1 Introduction 

To solve the stochastic control problems, there are two approaches: The dynamic 
programming method (HJB equation) and the maximum principle. 

In this paper, our system is governed by the stochastic differential delay 
equation (SDDE in short): 
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' dX{t) = b (t, X(t),Y(t), A(t),u(t)) dt 
+(j (t, X(t),Y(t), A(t),u(t)) dB{t) 

< + J Ro (t, X(t),Y(t), A(t),u(t), z) N(dt, dz); 
X(t)=X (t); 
Y(t) = X(t - S) 
A(t) = Jl s e-"^X(r)dr 
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t e 



t e 



t e 
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[0,oo) 
[0,oo) 



which maximise the functional 



J(u) = E f(t,X(t),Y(t),A(t),u(t)) 



dt 



(1.2) 



.0 



where u{t) is the control process. 

The SDDE is not Markovian so we cannot use the dynamic programming 
method. However, we will prove stochastic maximum principles for this problem. 

A sufficient maximum principle in infinite horizon whitout with non-trivial 
transvertility conditions where treated by [4] . The 'natural' transversality condi- 
tion in the infinite case would be a zero limit condition, meaning in the economic 
sense that one more unit of good at the limit gives no additional value. But 
this property is not necessarily verified. In fact [5] provides a counterexample 
for a 'natural' extension of the finite-horizon transversality conditions. Thus 
some care is needed in the infinite horizon case. For the case of the 'natural' 
transversality condition the discounted control problem was studied by [7] . 

In real life, delay occurs everywhere in our society. For example this is the 
case in biology where the population growth depends not only on the current 
population size but also on the size some time ago. The same situation may 
occur in many economic growth models. 

The stochastic maximum principle with delay has been studied by many 
authors. For example, [3] proved a verification theorem of variational inequality. 
[TO] established the sufficient maximum principle for certain class of stochastic 
control systems with delay in the state variable. In [JJ they studied inifinitc 
horizon bu withou a delay. In [2] , they derived a stochastic maximum principle 
for a system with delay both in the state variable and the control variable. 
In [12] they studied the finite horizon version of this paper, however, to our 
knowledge, no one has studied the infinite horizon case for delay equations. 

In our paper, we establish two sufficient maximum principles and one nec- 
essary for the stochastic delay systems on infinite horizon with jumps. 

For backward differential equations see [16], [6] For the infinite horizon BSDE 
see [M], [13], [17], [I] and [15]. For more details about jump diffusion markets see 
[TT] and for background and details about stochastic fractional delay equations 
see 0. 

Our paper is organised as follows: In the second section, we formulate the 
problem. The third section is devoted to the first and second sufficient maxi- 
mum principles with an application to the optimal consumption rate from an 
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economic quantity described by a stochastic delay equation. In the fourth sec- 
tion, we formulate a necessary maximum principle and we prove an existence 
and uniqueness of the advanced backward stochastic differential equations on 
infinite horizon with jumps in the last section. 

2 Formulation of the problem 

Let (CI, T , P) be a probability space with filtration Ft satisfying the usual condi- 
tions, on which an R-valued standard Brownian motion B (.) and an independent 

compensated Poisson random measure N(dt, dz) — N(dt, dz) — v (dz) dt are 
defined. 

We consider the following stochastic control system with delay : 
f dX(t) = b (t, X(t), Y(t), A(t), u{t)) dt + a (t, X(t),Y(t), A(t), u(t)) dB(t) 
+ Je(t, X(t),Y(t), A(t), u(t), z) N(dt, dz); t e [0, oo) 



R 



X(t) = X (t); 
Y(t) = X(t - S) 



t e [-S, o] 



A(t) 



-p(*-r)x(r)dr 



t-s 



(2.1) 



S > 0, p > are given constants. 
d:[0,oo)xlxRxMxMx!!^R, 
it:[0,oo)xKx1x1xWx!]^1, 
S:[0,oo)xRxlxIxWxloxS!4l, 

are given functions such that for all t, b(t, x, y, a, u, .), a(t, x, y, a, u, .) 
and 6(t, x, y, a, u, z.) are J^-mesurable for all x e R, y G K, a e M, u e U and 

z e R . 

Let £ t C J( be a given subfiltration, representing the information available 
to the controller at time t. 

Let Wbea non-empty subset of R. We let Ae denote the family of admissible 
£t-adapted control processes. 

An element of As is called an admissible control. 

The corresponding performance functional is 



J(u) = E 



OO 

Jf(t,X(t), 



Y(t), A(t), u(t)) dt 



;u e As, 



(2.2) 



where we assume that 



OO 

eJ \\f(t,X{t),Y{t),A(t),u{t)) 



^-(t,X(t),Y(t),A(t),u(t)) 



dt < oo 
(2.3) 
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The value function $ is defined as 

$(X ) = sup J{u) (2.4) 

An admissible control u* (.) is called an optimal control for (2.1) if it attains 
the maximum of J (u (.)) over As- (2.1) is called the state equation, the solution 
X* corresponding to u* (.) is called an optimal trajectory. 

3 A sufficient maximum principle 

Our objective is to establish a sufficient maximum principle. 

3.1 Hamiltonian and time-advanced BSDEs for adjoint 
equations 

We now introduce the adjoint equations and the Hamiltonian function for our 
problem. 

The Hamiltonian is 



H(t, x, y, a, u,p, q, r(.)) = f(t, x, y, a, u) + b(t, x, y, a, u)p + a(t, x, y, a, u)q 

6(t, x, y, a, u, z)r{z)v{dz) 1 



R 

(3.1) 

where 

i7:[0,co)xRxKxlx[/xIRxlx$xn^l 

and 5ft is the set of functions r: Ro — > R such that the terms in (3.1) converges 
and U is the set of possible control values. 

We suppose that b, a and 9 are C 1 functions with respect to (x, y, a, u) and 
that 



E 



^-(t,X(t),Y(t),A(t),u(t)) 



89 



^(t,X(t),Y(t),A(t),u(t)) 



— (t,X(t),Y(t),A(t),u(t)) 



v(dz) > dt 



< oo 



(3.2) 



for Xi = x, y, a and u. 

The adjoint processes (p(t), q(t), r(t, z)), t 6 [0, co), z G R are assumed to 
satisfy the equation : 
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dp(t) = E \fi{t) | T t ] dt + q(t)dB t + J r(t, z)N(dt, dz); t e [0, oo) , (3 3) 

R 



where 



3H 

M*) = —fa (t'XuYt, A t , u t ,p(t),q(t),r(t,.)) 



dH 
dy 



(t + S, X t+s ,Y t+ 8, A t+S , u t+s ,p(t + 5), q(t + S),r(t + 5, .)) 



' t+s 



s,X s ,Y s , A s , u s ,p(s),q(s),r(s, .)) e p8 ds 



(3.4) 



3.2 A first sufficient maximum principle 

Theorem 3.1 Let it G As with corresponding state processes X(t), Y(t) and 

A A 

A(t) and adjoint processes p(t), q(t) and f(t,z) assumed to satisfy the ABSDE 
(3.3)- (3.4). Suppose that the following assertions hold: 

ft) E 



lim p(t)(X(t) - X(t)) 

t— >oo 



> 0. 



(ii) The function 



(x,y,a,u) ->■ H(t,x,y,a,u,p,q,f(t.)), 



is concave for each t G [0, 00) a.s. 
(Hi) 



E 



p 2 (t) a 2 (t) + 6 2 {t,z)v{dz) 



+X 2 (t) lq 2 (t)+ Jr{t,z)v{dz) \ \dt 



< 00, 



for all u G As- 

(an) 



vaaxE 



E 



H(t, X(t),X(t - S), A(t),v,p(t), q(t), r(t, ■) I £t 
H(t, X(t),X(t-S),A(t),u(t),p(t),Q(t),r(t,.)\£t 



for all t G [0, 00) a.s. 

Then u{t) is an optimal control for the problem (2.4). 



(3-5) 
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Proof. Choose an arbitrary u e Ae , and consider 

J(u) - J(u) = h 

where 



(3.6) 



/ (t, X(t), Y(t), A(t), «(t)) -fit, X(t), Y(t), A(t), u(t) dt 



h = E 



By the definition (3.1) of H and the concavity, we have 



(3.7) 



L < E 



pifj f)M f)M 

— (t)(X(t) X(t)) + -gj(t)(Y(t) - Y{t)) + -^(t)(A(t) A(t)) 



oh 



+ -g^(t)(u(t) «(*)) (b(t) b(t))p(t) (a(t) a(t))q(t) 



-J (6(t,z) - §(t,z))f(t,z)u(dz) 



dt 



(3.8) 



where we have used the simplified notation 

-fa{t) = (t,X t ,Y t , A t , u t ,p{t),q{t),r(t,.)) 
Applying the Ito formula to p(t)(X(t) — X(t)) we get 



< E 



= E 



lim p(T)(X(T) - X(T)) 

T->oo 

T 



lim 



J (b(t) - b(t))p{t)dt + J (X{t) - X(t))E [Mi) | F t ] dt 
o o 

T 

J \a{t) - a(t))q(t)dt + J j \o(t, z) - 6(t, z))f(t, z)v{dz)dt 



R 



E 



(J- * 
J (b(t) - b{t))p{t)dt + J (X(t) - X{t))(i{t)dt 



{a{t) - a(t))q(t)dt 



+ J J (6(t, z) - 0(t, z))f(t, z)v{dz)dt 

R 



(3.9) 



Using the definition (3.4) of /i we see that 

V ~ 









E 


lim 


;/ 




T^oc 








\0 



(X(t) - X{t))fi{t)dt 



= E 



' T+S 



lim 

T->oo 



(X(t -5)- X(t - S))fi(t - 5)dt 



E 

T+S 



T+S 



lim | - 



OH 

dx 



(t - 5){X{t - 5) - X(t - 8))dt 



f dH . , 



T+S / t 



(Y(t) Y{t))dt 



dH 

da 



(s) e~P s ds 



t-S 



e p( t - s )(X(t-5)-X(t-6)f) dt 
Using integration by parts and substituting r = t — 5, we obtain 

T „ 

fdH a 
j—(s)(A(s)-A(s))ds 

o 

T „ s 

= J^(s) J ' e- p{s - r) {X(r) - X(r))dr ds 
o s-a 

— (s)e-" s ds e" r (X(r) - X(r)) dr 
^ (s) e~P s ds j e^*" 5 ) (X(t - 5) - X(t - <y))dt 

5 V-<5 / 

Combining (3.9), (3.10) and (3.11); we get 



(3.10) 



T / r+S 



\ r 
T+S / t 



(3.11) 



< E 



lim p(T)(X(T) - X(T)) 

.1 — >oo 



E 



L \0 



(b(t) - b(t))p(t)dt 
-J H (*) (*(*) - Xit))dt " /^f (*) W f {t))dt 



•dH 



-J±-(t)(A(l)-Ml»,ll+ liMD-alDujiluIl 



CO > 

J J {6{t, z) - §(t, z))f(t, z)v{dz)dt 



R 



(3.12) 



dH , 



Subtracting and adding / ^-(i)(u(i) - u{t))dt in (3.12) we conclude 



< E 

OO 

'dH 
dx 



lim p(T)(X(T)-X(T)) 

1 —too 



E 



(b{t) - b(t))p(t)dt 



(t,X(t),Y(t), A(t),u(t),p(t),q(t), r(t, .)) (X(t) - X(t))dt 



o 

OO 



dH 

— (t,X(t),Y(t), A(t), u(t),p(t),q(t), r(t, .)) (Y(t) - Y(t))dt 
ay 

— (t, X(t), Y(t), A(t), u{t),p{t),q{t), r(t, .)) (A(t) - A{t))dt 



(a{t) - a(t))q(t)dt + j \ (6(t, z) - 6(t, z))r(t, z)v(dz)dt 
o o 



dH 

du 



(t)(u(t) - u{t))dt 



dH 

du 



(t)(u(t) - u{t))dt 



<-h+E 



E 



dH 

du 



(t)(u(t)-u(t))\£ t 



Hence 



h < E 



E 



(u(t) - u{t))dt 



dt 



< 0. 



Since u £ As was arbitrary, this proves Theorem 1. ■ 

3.3 A second sufficient maximum principle 

We extend the result in |10j to infinite horizon with jump diffusions. 
Consider again the system 

' dX{t) = b{t,X(t),Y(t),A(t),u(t))dt 
+0- (t, X(t),Y(t), A(t),u(t)) dB{t) 

+ J Rq 9 (t, X(t),Y(t), A(t),u(t), z) N(dt, dz); t€ [0, oo) 

X(t)=X (t); t€[-8,0] 

Y{t)=X{t-5) t€[0,oo) 

A{t) = Jl_ s e~ p ^X(r)dr t € [0, oo) 

Let X t G C[— 8, 0] be the segment of the path of X from t — 6 to t, i.e 

X t (s)=X(t + s), 
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for s £ [—5,0]. We now give an Ito formula which is proved in [3] without 
jumps. Adding the jump parts are just an easy observation. 

Lemma 3.2 The ltd formula for delay 
Consider a function 

G(t)=F(t,X(t),A(t)), (3.13) 

where F is a function in C 1:2,1 (R 3 ) and 

,o 

Y(t) = J _e Xs X(t + s)ds. 

then 
Then 

dF 

dGit) = LFdt + ait, x, y, a, u) —dB(t) 

ox 

f.F(t, X{r), A(t~)) + 6{t, X(t),Y{t),A(t), u, z) 

-F(t,X(t-),A(t-)) 

dF ) 

■ — (t, X(t-),A(t~)) + 0(t, X(t),Y(t),A(t), u, z) \v{dz)dt 
ox I 

+ I {F(t,X(r),A(t-))+6(t,X(t),Y(t),A(t),u,z) 

-F(t, X{r), A{t-))^N{dt, dz) 

+[x - Xy - e- xs a]—dt 
oa 



where 



rr, rr,, OF 8F 1 J 2 f 

LF = LF(t, x, y,a,u) = — + b— + -a 2 



as 



dt dx ' 2 dx 2 
Now, dehne the Hamiltonian, H:R + xRxRxRx[/xR 3 xR 2 x7?.^ xM 

H(t,x,y,a,u,p,q,r(-)) (3.14) 
= f(t, x, y, a, u) + b(t, x, y, a, u)p\ + (x - Xy - e~ xs a)p 2 

+ a(t,x,y,a,u)qi + / 9(t,x,y,a,u, z)r{z)v(dz) 

JS.0 

where p = (pi,P2,P3) T S R 3 and q = (qi, q 2 ) 6 R 2 For each u e A the associated 
adjoint equations are the following backward stochastic differential equations in 
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the unknown ^-adapted preocesses q{t),r(t, •)) given by; 
OH 

dpi(t) = —g^(t, X(t), Y{t),A(t), u(t),p(t), q(t))dt + qi (t)dB(t) 



+ / r(t,z)N(dt,dz), 

JRn 



dp 2 (t) = - — (t,X(t),Y(t),A(t),u(t),p(t),q(t))dt + q 2 (t)dB(t), (3.15) 
oy 



dp 3 (t) = —^-(t,X(t),Y(t),A(t),u(t),p(t),q(t))dt, 



(3.16) 



Theorem 3.3 (An infinite horizon maximum principle for delay equations) 

Suppose u e A and let (X, Y, A) and (p(t), q(t),r(t 7 •)) be the corresponding so- 
lutions of (3.15)-(3.16), repectively. Suppose that 



H(t,;;;;, P (t),q(t),r(t,-))) 



are concave for all t > 0, 



E 



= maxi? 



Further, assume that 



and 



H(t, X(t),Y(t), A(t),u(t),p(t), q(t), f(t, -))\£ t 
H(t, X(t), Y(t), A(t), u,p(t),q(t),f{t, -))\£t 

E[\imp\(t)(X(t) - X(t))] >0, 
E[\\mp 2 {i)(Y{t) - Y{t))\ > 0. 



In addition assume that 

Pa(t) = 0, 

for all t. Then u is an optimal control. 
Proof. To simplify notation we put 

((t) = (X(t),Y(t),A(t)), 

and 



Let 



((t) = (X(t),Y(t),A(t)). 



:= E[ / (f(t, C(t), fi(t)) - f(t, C(t), u(t)))dt) 
Jo 



(3.17) 

(3.18) 
(3.19) 
(3.20) 
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Then we have that 



poo 

I = E[ (H(t, C(t), u(t),p(t), q(t),r(t, ■)) - H(t, ((t),u(t)),p(t),q(t), r(t, -))dt] 
Jo 

poo 

- E[ / (b(t, C(t),u(t)) - b(t, C(t),u(t))) Pl (t)dt] 

Jo 

- E[ / {(X(t) - XY{t) - e- x5 A{t)) - (X(t) - XY(t) - e~ xs A{t))}p 2 {t)dt] 

Jo 

r-OO 

- E[ / {<r(t, C(t), «(*)) - <r(t, C(t), u(t))}«i (t)dt] 

Jo 

-E[ / (e(t,C(t),u(t),z)-6(t,C,u,z))xr(t,z)v(dz)di\ 

JO JMo 

=:h+h+h+h + h- (3.21) 
Since ((, u) — > H((,u) is concave and (3.12), we have that 

H(C, u) - H(C, u) < H ( (C fi) • (C - C) + ff u (C, 6) • (« - fi) 

<ff c (C,fi)-(C-C) 

where £T C = (||, §f , |f ). From this we get that 



L > E 



-H c (t, C(t), u(t),p(t), 9(0) ' (C(*) - C(*))d* 



t / (C(i) - C(t))dp(*) - / (*(*) - X(t)) qi (t)dB(t) 

pOO 

(Y(t)-Y(t))q 2 (t)dB(t) 

Jo 

/•oo />oo 

= £ / (X(t)-X(t))d Pl (t) + (Y(t)-Y(t))dp 2 (t) 
Jo Jo 

From (3.18), (3.19) and (3.20) we get that 

> -E[]imp\(t)(X(t) - X{t)) + \imp 2 (t)(Y(t) - Y{t))} 

pOO pOC 

= -E / (X(t) - X(t))d Pl (t) + / Pl (t)d(X(t) - X(t)) 
Jo Jo 

+ a(t,((t),u(t))-a{t,((t),u(t) qi (t)dt 
Jo 1 J 

+ / I {6(t,((t),u(t),z)-e(t,<;,u,z))xr(t,z)is(dz)dt 
Jo Jr 

p OO f-OO 

+ (Y(t)-Y(t))dp 2 (t) + p 2 (t)d(Y(t)-Y(t)) . 
Jo Jo 



(3.22) 
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Combining this with (3.21) and (3.22) we have that so that 
-I = Ji + 1 2 + I 3 + h + h < 0. 

Hence J{u) — J{u) = I > 0, and u is an optimal control for our problem. 
■ 

Example 3.4 (A non-delay infinite horizon example) Let us first consider 
a non-delay example. Assume we are given 



7 G (0, 1) and p,8 > 0. In this case the Hamiltonian (3.14) takes the form 




where 



dX(t) =[X(t)n-u(t)X(t)]dt 

+a(t, X{t),u{t))dB{t);t > 
X(t) = X Q 




so that we get the partial derivative 



V u H(t,u,x,p,q) = e pt u< 1 x 1 - xp 1 - — q l . 



This gives us that 



p^^e-^x^uitr- 1 -^- 



We now see that the adjoint equations are given by: 



d Pl (t) = - e-p'iu^rxity- 1 



+ u(t)) Pl (t) + p 2 (t) + — qi (t) J dt + qi (t)dB(t), 
dp 2 (t) = -q 2 (t)dB(t), 

d P3 (t) = e- pS p 2 (t) + -^qi(t) dt. 



Since Pz{t) must be 0, we then get qi = q 2 = 0. and 



P2(t) = 0, 
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which gives us that 



d Pl (t) = - [e-f^u^X^^dt + u(t))pi(t))] dt, 
d P2 (t) = 0, 



Pl {t) = e-» t x{ty- 1 u(ty- 1 



So 



d Pl (t) = ~ [e-^iuWXity^dt + u(t))pi(t)] dt, 
= —fj,pi(t)dt 

which gives 

Pi(t)=Pi(0)e-" t , 
for some constant pi(0), so that 

for all t > 0. Inserting u into the dynamics of X , we get that 



dX(t) = \fiX(t) -p^O^e^^-rt 



dt. 



So 



X{t) = e"* 



X(0)- Pl (0)— / exp((-fi-- (X-fx))s)ds 

o 1 ~ 7 



To ensure that X(t) is alwasys non-negative, we get the optimal p(0) as 

1 7-1 

\ (1)1 

Pi(0) 



X(Q) 



We now see that \impi(t) — 0, so that we have 

E[\imp\(t)(X(t) - X(t))] > 0. 
This tells us that u is an optimal control. 

Example 3.5 (An infinite horizon example with delay) Now let us con- 
sider a case where we have delay. This is an infinite horizon version of Example 
1 in JTUj/. Let 



J(u) = E 



e- pt -(u(t)(X(t) + Y(t)e p5 p))~<dt 
o 7 



13 



where 

r dX(t) = [X(t)n + Y(t)a + /3A(t) - u(t)(X(t) + Y(t) e P s f3)]dt 

+a(t, X (t),Y(t), A(t),u(t))dB(t);t > 0, 
X(t) =X (t);t€[-S,0], 

7 G (0, 1) and p, 5 > 0. In this case the Hamiltonian (3.20) takes the form 

H(t,u,x,y,a,p,q) = e~ pt -(u(x + ye pi ; /3)) 7 + [x/j, + ay + /3a - u(x + ye pS (i)]pi 

7 

+ [x - Xy - e~ p5 a]p 2 + a(t, x, y, a, u)q, 
so that we get the partial derivative 

\7 u H(t,u,x,y,a,p,q) = e-P t u~<- 1 (x + yeP 5 (3y - (x + ye pS (3) Pl - 
This gives us that 

M^e^ix + ye^r-M^-^^^f, 

We now see that the adjoint equations are given by: 
dpi(t) = -[e- pt {u{t)y(X(t) +Y{t)e p5 f3y- 1 dt 

+ (m - u(t)) Pl (t) + P2 (t) + -£qi(t)]dt + qi {t)dB{t), 

dp 2 (t) = -[e- pt {u{t)^(X(t)+Y(t)e ps /3y- 1 e ps pdt 

+ (a- u{t)e pS /3)pi(t) - Xp 2 {t) + -^qi(t)]dt + q 2 (t)dB(t), 

dp 3 (t) = -[(3 Pl (t) - e- p5 p 2 (t) + ^qi(t)]dt. 

Let us try to choose qi = q 2 = 0. Since p?,{t) = 0, we then get 

- P S 



Pi(*) = -jg-P2(*), 



which gives us that 



dpi(t) = -[e- pt (u(t)r(X(t) + Yifyeftpy^dt + (fx - u(t)) Pl (t) + e ps (3 Pl (t)]dt, 
dp 2 (t) = -[e- pt (u(t)y(X(t) + YtDe^Py-Wpdt + (a^ - u(t))p 2 {t) - Xp 2 (t)]dt, 



and 

Pl (t) = e-"*(x(t) + y(t) e " 5 /3) 7 - 1 u (t) 7 - 1 
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or 



Hence, to ensure that 
we need that 

a = e ps /3{ h i + X + e ps (3). 

So 

dpi(t) = -[e- pt (u(t)y{X{t) + Y{t)e p5 py- X dt + (p - u{t)) Pl (t) + e ps p Pl (t)]dt, 
= -^p 1 (t)+e ps l3p 1 (t)]dt 7 

which gives us that 

for some constant pi(0) . Hence by (3.23) we get 

[) Pl(0) (X(t)+Y(t)e pS P) h 

for all t > and some pi(0). In analogy with Example 3.4 it is natural to 
conjecture that the optimal value, K, of pi(0)is given by 

K = inf{pi(0) : X Pl(0) (t) + Y Pl{0) (t)e xs (3 > 0, for all t > 0}, 

see J2|/. So, the optimal control is given by 

i 

U[t) ~ (X(t) + Y(t)e p *f3) e >■ 
From this we get that limpi(i) = \imp2(t) = 0, so that we have 
Epimft(t)(A:(t) - X(t))} > 0, 

and 

E[limp 2 (t)(Y(t) -Y(t))} > 0. 
This tells us that u is an optimal control. 
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4 A necessary maximum principle 

In addition to the assumptions in the previous section, we now assume the 
following. 

(Ai) For all u G As and all j3 G As bounded, there exists e>0 such that 

u + s/3 G As for all s G (— e, e). 

(A 2 ) For all to, h and all bounded £ to -mesurable random variables a, the 
control process j3(t) defined by 

(}(t) = al [s , s+h] (t) (4.1) 

belongs to As- 

(A3) For all bounded (3 G As, the derivative process 

£(t) := ±X»+>e(t) U (4.2) 

exists and belongs to L 2 (X x P). 
It follows from (2.1) that 

dm = ^ mt) + lTy {m - S) + £ W ]j- p(t - r) ^)dr + j dt 

^mt) + d ^mt -S) + ^(t) j e-"^(r)dr + ^-(t)P(t)\ dB(t) 

t-5 ) 

t 

' (t, z)Z(t) + —(t, z)£(t -5) + —(t, z) / e-P^- r ^(r)dr + — (t, z)/3(t) \ N(dt, dz), 



dx dy da J du 

t-5 

where, for simplicity of notation, we define 



Ji b{t ) := X(t),X(t 5),A(t), u(t)), 



and used that 



±Y»+'P{t) |. =0 = ^X u+Sfj (t S) U= £(t 5) 



and 
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^A«+*>(t) U= j- g ^e-*-r)X^(r)drj |. =0 

- ^Je-rt-'^Xv+'tir^j \ s=0 dt 
t 

= j e-P^- r ^(t)dr. 



Note that 



£(t) = for i e [-5, oo) . 



Theorem 4.1 (Necessary maximum principle) Suppose that u e As with 
corresponding solutions X(t) of (2.1)-(2.2) andp{t), q(t), and f(i, z) of (3.2)- 
(3.3), and corresponding derivative process £(i) groen 6y (4.2). 
Assume that for all u £ As the following hold: 



E 



i) «-* 2 «+(t;) (*■»>«'('-')+ (I) ; 



-p(*-»-)£(r)dr 



+ y < $(*) + j f 2 (t,z) v[dz) \ dt < 



oo. 



and 



E 



lim p{t){X{t) - X{t)) 

t— >oo 



> 0. 



Then the following assertions are equivalent, 
(i) For all bounded £ As, 



-J(u + S /3) U=0. 



(ii) For all t £ [0, oo), 



E 



H(t, X(t),Y(t),A(t),u(t),p(t),m,f(t,.)\£ t 



u—u{t) 



a.s. 
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Proof. Suppose that assertion (i) holds. Then 



= -J(u + s(3) U 



ds 



E 



f(t, X &+s ' 3 (t) 1 Y il + s ' 3 (t) 1 A ij '+ s f 5 (t) 1 u{t) + a 0{t)dt 



t-S 



df, 



We know by the definition of H that 

d f , . <9i? , . db ... . da ... . 



(9(9 
<9.r 



(t, ^)r(f, z)v(dz) 



df df df 
and the same for — — (f), — — (i) and — — (f). 

ay <?» cm 

We have 



lim p(t)(A"(t)-X(t)) 



> 



So 



lim 

t— >oo 



> £ 



lim 

t— >oo 



for all (3 E Ag and all s G (— e, e). 
Hence 



If 



d 



ds 



( lim p(t)X ii+sf:S (t)) L=o< ff(iu) , where is some integrable 

Vt— >oo / 

function. From the uniform limits with uniform convergence of the derivative, 
we can interchange the derivative and integration, and get 



= • 

= E 

= E 



^{hm (p^X^+^it)) 

A{ S (mx^it))} 



s=0 



\s=0 



s=0 ■ 
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Applying the Ito formula to f>(t) — (X" +s/3 (i)) wc obtain 



= E 



E 



^(*)e(t) + |£(*)£(t - + £w/e-^-^(r)dr + > ■/' 



+ 1 £(t)i3 (/*(*) | F t ) dt + J q(t) g(i)£(t) + ^(tW -S) + ^(t) J e-"^H(r)dr + ^(W) 

o I ts 

+ I I r(t, z) { g(t, + g(t, - <J) + |£(t, s) J e-rt-^{r)dr + |£(t, *)/J(t) 1 v{dz)dt 

t-s ) 



R 

= --^J(ii + s(3) | s=0 +£ 



Therefore 



, — - ^ 

/f WW* 

/ oo ^ 



Use 



= al [SiS+h] (t) 

where a(w) is bounded and £ to -mesurable, s > to and get 



' s+h \ 

J^(s)dsaj =0 



Differentiating with respect to ft- at 0, we have 
This holds for all s > to and all a, we obtain that 



(t )\£ to )=0. 
This proves that assertion (i) implies (ii). 

To complete the proof, we need to prove the converse implication; which is 
obtained since every bounded /3 £ As can be approximated by linear combina- 
tions of controls /? of the form (4.1). ■ 
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5 Existence and uniqueness of the time-advanced 
BSDEs on infinite horizon 



The main result in this section refer to the existence and uniqueness for (3.3) — 
(3.4) where the coefficients satisfy a Lipschitz condition. 

We now study time- advanced backward stochastic differential equations driven 
by a Brownian motion B (t ) , and a compensated Poisson random measure N(dt,dQ. 

Let B(t) be a Brownian motion and N(dt, d() :— N(dt, d()—i/(d()dt, where v 
is the Levy measure of the jump meaure iV(-, •), be an independent compensated 
Poisson random measure on a filtered probability space (f2, F, {F t }o<t<oo}- 

Given a positive constant 5, denote by D([0, S],M.) the space of all cadlag 
paths from [0, 5] into R. For a path X(-) : M. + — > K, X t will denote the function 
defined by X t (s) = X(t + s) for s G [0,(5]. Put H = L 2 (v). Consider the L 2 
space Vi := L 2 ([0, 5} ->• R; ds) and V 2 := £ 2 ([0, S] H; ds). Let 



F 



xRxRxV 1 xRxRxV 1 xHxHxV 2 xil 



be a function satisfying the following Lipschitz condition: There exists a con- 
stant C such that 

I F (t, Pi, P2,P, 9i, 92, q,n,r 2 ,r,co) - F(t,p 1 ,p~ 2 ,p,q 1 ,q 2 ,q, ri,f 2 ,f,u)\ 
< C(\pi - pi\ + \p 2 - p 2 \ + \p-p\v 1 + \qi - qi\ + |<?2 -q 2 \ + \q- q\ Vl 
+ \n -fi\ H + \r 2 -%\ n + \r- f\ Va ). (5.1) 

Assume that (t,cj) — > F(t,pi,p 2 ,p,qi,q 2 ,q,ri,r 2 ,r,u) is predictable for all 
Pi,p 2 ,p, qi,q 2 ,q, ri,r 2 ,r. Further we assume the following: 

/•OO 

E / e At |F(t,0,0,0,0,0,0,0,0,0)| 2 dt < oo, 



for all A G M. We now consider the following backward stochastic differential 
equation in the unknown ^-adapted processes (p(t),q(t), r(t, z)) 6 H x H xH: 

dp(t) = E[F(t,p(t),p(t + S,p t ,q(t + 5), q t ,r(t + 5), r t )\F t ]dt 



+ 



where 



for all A 6 



q(t)dB(t)+ r(t,z)N(dz,dt) 

f 

Jo 



\p(t)\ z dt 



< oo, 



(5.2) 



(5.3) 



Theorem 5.1 (Existence and uniqueness) Assume the condition (5.1) is 
fulfilled. Then the backward stochastic partial differential equation (5.2) - (5.3) 
admits a unique solution (p(t),q(t),r(t, z)) such that 



E 



f 

Jo 



-M. 



{\p(t)\ 2 + \q(t)\ 2 + / \r(t,z)\ 2 v(dz)}dt 



< oo, 



for all A e 
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Proof. 
Step 1: 

Assume F is independent of its second, third and fourth parameter. 

Set q°(t) := 0, r°(t,z) := 0. For n > 1, define {p n (t),q n {t),r n (t,z)) to be 
the unique solution of the following BSDE: 

dp n (t) = E [F(t, q n -\t), q n -\t + 5), q?-\r n -% .),r n -\t + 8, .), x (.)) \ Ft] dt 
+ q n {t)dB{t)+ [ r n {t 1 z)N{dt 1 dz)- (5.4) 

for t e [0, oo) such that 



E 



e xt \p n (t)\ 2 dt 



< oo 



This exists by Theorem 3.1 in [3]. 

Our goal is to show that (p™ (t) , q n (t) , r n (t, z)) forms a Cauchy sequence. 
By Ito's formula we get that 



= |eV +1 W-/WI'+ / \e AS \p n+1 (s)-p n (s)\*ds 



„"/„\|2, 



e As \q n+i {s) -q n (s)\ A ds 



+ / e As / \(r n+1 (s,z) ~r n (s,z))\ 2 dsv(dz) 



+ 2^ e 
"I 6 



Xs ((p n+1 (s) -p n (s)) ,E[F n - F"- 1 | F s ])ds 

Xs (p n+1 (s) ~p n (s)) (q n+1 (s) - q n ( S ))dB s 

: I I e Xs (\r n+1 ( S ,z)-r n (s,z)\ 2 
It Jr 

+ 2(p n+1 ( S -)-p n ( S -)) (r n+1 (s,z)~r n (s,z)))N(ds,dz). 
Rearenging, using that for all e > 0, ab < — + eb we have by the Lipschitz 
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requirement (5.1) 

E[e xt \p n+1 {t)-p n {t)\ 2 ] 

/CO 
Ae A > n+1 (s) -p n (s)\ 2 ds 

/CO 
e Xs \q n+1 (s)-q n (s)\ 2 ds] 

+ E[H f e Xs \(r n+1 (s,z)-r n (s,z))\ 2 V (dz)ds} 

/oo 
e As |p n+1 (s) -p n {s)\ 2 ds 

/CO 
e Xs \q n {s)-q n - 1 (s)\ 2 ds] 

/CO 
e Xs \q n {s + 6)-q n - 1 (s + 6)\ 2 ds] 

/■oo / , s+<5 

+ e6£[ / e Xs / |g n (u) - g^ujfduda] 

Jt J S 

/CO 
e As |r"(s) -r^OO&ds] 

/CO 
e As |r n (s + 6) -r^s + ^ds] 

/■CO /"S+fJ 

+ e6E[ e Xs \r n (u) - r"" 1 {u)\ 2 H duds\ 

Jt J s 

where C e = ^- and we used the abbreviation 

F n (t) := F{t, q n {t), q n (t + 5),q?,r n (t, .),r n (t + 5, .),!?(•)). 
Note that 



/CO 
e As |g"(s + c5) -^(a + djfds 

/CO 
e As |g"(s)-g"- 1 (s)| 2 ds]. 
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Using Fubini 

»oo rs-\-S 



/oo rs+o 
J e A '|« n (u)-« n - 1 (u)| 2 di«i8 

pOO pU 

<E[ / e Xs \q n {u) -g n - 1 («)| 2 dad« 

Jt Ju-S 

<(-(l-e-™)E[J t e x *\q n (s)-q n -\s)\ 2 ds] 

/OO 
e Xs \q n {s)-q n - 1 (s)\ 2 ds]. 

Similiar for r n — r n_1 . It now follows that 
Ee xt \p n+I (t)-P n (t)\ 2 } 

/OO 
e Xs \q n+1 ( S )-q n (s)\ 2 d S } 

+ E[f°° f e Xs \(r n+1 (s,z)-r n (s,z))\ 2 is(dz)ds} 

Jt J M 

/OO 
e Xs \p n+1 {s) -p n {s)\ 2 ds] 

/OO 
e Xs \q n (s)-q n - 1 (s)\ 2 ds] 

/OO 
e Xs \r n (s) -r n - 1 (s)\ 2 n ds}. (5.5) 

Choosing e = 12 (2+e-* s ) so ^at 
Se At |p Il+1 (t) -p"(t)| 2 ] 

/OO 
e Xs \q n+1 ( S )-q n (s)\ 2 ds] 

+ E[[°° f e Xs \(r n+1 ( Sl z) - r n {s,z))\ 2 v{dz)ds] 

Jt Jr 

/OO -I fOO 

e Xs \p n+1 ( S )-p n ( S )\ 2 d S ] + -E[ J e Xs \q n ( S ) - q^^fds] 

+ -E[J^ e Xs \r n {s)-r n -\s)\ 2 i ds] 
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This implies that 

Q roo 

-(e^-^El J e Xs \p n+1 (t) -p n (t)\ 2 } 

/OO 
e Xs \q n+1 (s)-q n (s)\ 2 ]ds 

+ e (c.-\)t E \ r f e ^\(r n + 1 ( a ,z)-r n (a,z))\ ,i u(dz)ds] 
Jt Jr 

1 f°° 

< e (C.-A)t_g[ / e A.| g n^ _ 9 «-l( s )|2 ds 

2 Jt 

1 Z" 00 

+ i e (C e -A)t^ / e A»| r n^_ r"- 1 ( S )|^ds 

2 Jt 
Integrating the last inequality we get that 

/•OO 

£[/ e As |p" +1 (t) -p n {t)\ 2 dt) 
Jo 

/■ oo />oo 

+ / e^- X ^E[ e Xs \q n+1 (s)-q n (s)\ 2 ds} 
Jo Jt 

/*oo /*oc /* 

+ / e {c ^ x)t E[ / e As |(r" +1 (s,z) - r"(s, z))| 2 i/(dz)da]dt 

JO Jt JR 

-i /*oo />oo 

< x / e (c '- A) *S[ / e Xs \q n {s) - q n - 1 {s)\ 2 dsdt 
2 Jo Jt 

-I />oo />oc 

-/ e^~ X ^E e Xs \r n (s)-r n - 1 (s)\ 2 H dsdt. 
2 Jo Jt 



+ 2 



So that 



e (C«-A)t^j / e A S | g „+l (s) _ g «( s )|2 ds ] 
Jt 



/•OO roo p 

+ / e {C '- x)t E[ / e As |(r" +1 (s,z) -r"(s,z))| 2 ^(dz)ds]^ 

JO Jt JR 

1 /'OO /'OO 

<x/ e^ C '- X ^E[ e Xs \q n (s)-q n - 1 (s)\ 2 dsdt 

2 Jo Jt 

-i />oo />oo 

+ ^ / e^- x ^E e Xs \r n (s)-r n - 1 {s)\ 2 n dsdt 
2 Jo Jt 
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This gives that 



e 

poo 



/ e {C '- x)t E[ / e Xs \(r n+1 (s,z) - r n (s, z))\ 2 v(dz)ds]dt 

JO Jt JR 



< — c 3 , 

if A > y- It then follows from (5.6) that 

POO -I 

£[jf e Ai |p" +1 (t)-/(t)| 2 ]<-C 3 . 

From (5.5) and (5.6), we now get 

E[f f e Xs \(r n+1 (s,z) - r n (s, z))\ 2 v(dz)ds]dt 
Jt Jr 

/oo i 
e Xs \q n+1 (s)-q n (s)\ 2 ds} < -C 3 nC e . 

From this we conclude that there exist progressively measurable processes 
(p(t),q(t),r(t, z)), such that 



lim E 

n— >oo 


[e At b"(t)-p(t)| 2 dt] 


= 0, 


/•OC 

lim E{ / 


e As b"(t)-p(t)| 2 dt] 


= 0, 


/>oo 

lim £[ / 


e Xs \p n (t)-p(t)\ 2 dt] 


= o, 


/•OO 

lim £[ / 

rwoc J Q 


e Xs \p n {t)-p{t)\ 2 dt] 


= 0, 


e As |(r"( S ,z)- 


- r(s, z))'\ 2 v(dz)ds\dt 


= 0. 



lim E[ 

n— t-oo 

Letting n — > oo in (5.4) we see that (p(t),q(t),r(t, z)) satisfies 

dp(t) - £ [F(t, q(t),q(t + 6), q t ,r{t, .), r(t + S, .), r t (-)) I ^t] dt 
+ q(t)dB(t)+ r(t,z)N(dt,dz), 

JEo 

for all i > 0. 

Step 2: 

General F. 
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Let p°(t) = 0. For n > 1 define (p n (t),q n (t),r n (t, z)) to be the unique 
solution of the following BSDE: 

dp n (t ) = E [F (t , p n ~ 1 (t ) , p n ~ 1 (t + 8) , P r 1 , q n (t ) , g" (t + S) , , r" (t ) , r n (t + 6) , r? ) 1 7i] dt 
+ q n {t)dB{t)+ [ r n (t,z)N(dz,dt), 

for i e [0, oo). The existence of (p n (t), q n (t), r n (t, z)) was proved in Step 1. 
By using the same arguments as above, we deduce that 

Ee Xt \p n+1 (t) -p n (t)\ 2 } 

/OO 
e Xs \q n+1 (s)-q n (s)\ 2 ds} 

+ E[[ X e Xs f \(r n+1 (s,z)~r n (s,z))\ 2 v(dz)ds} 

Jt J M 

/OO -t pOO 

e Xs \p n+1 ( S )-p n ( S )\ 2 d s } + -E[ J e Xs \p n (s)-p n - 1 (s)\ 2 ds] 



This implies that 

j /■ OO i r-oo 

--{e {c '- x ^E[ e Xs \p n+1 (s) -p n {s)\ 2 ds}) < -e<- c ^ x ^E[ e Xs \p n {s) - p^a^ds]. 
dt Jt 2 J t 

Integrating from to oo, we get 

pOC -I r-OO r-OO 

E[ / e Xs \p n+1 (s) -p n (s)\ 2 ds] <- e^'-^Ei / e Xs \p n (s) - p^ 1 {s)\ 2 ds\dt 
Jo 2 Jo J t 

So if A > C e then by iteration we see that 

E[J Q e x °\p^(s)-p«(s)\ 2 ds}< 2n{x _ cdn , 

for some constant K. 
Uniqueness: 

In order to prove the uniqueness, we assume that there are two solutions (p 1 (t), q 1 (s), r 1 (s, z 
and (p 2 (t),q 2 {s),r 2 {s,z)) of the ABSDE 

dp(t) = E [F(t,p{t),p(t + S),p t ,q{t),q(t + <5), q t ,r(t), r(t + S),r t ) \ T t \ dt 
+q(t)dB(t) + j r(t,z)N(dt,dz); t e [0, oo) 



E 



I e xt \p(t)\ 2 dt 



Lo 



< oo ; A E 



By Ito's formula, we have 
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E 

+E 



,A t |„1 



p\t)-p 2 {t)\ 



E 



J A e Xs \p 1 (s) -p 2 (s)\ds 



J e Xs \q 1 ( S )-q 2 (s)\ 2 ds 



+ E 



J e x s J jr^s, z) - r 2 (s, z)\ dsv(dz) 

_t R 

= 2Eje Xs [ \p 1 {s)-p 2 {s)\ 

x (E [ F(s,p 1 (s),p 1 (s + S),p 1 s ,q 1 (s),q 1 (s + S) , q\ , r 1 (s) , r 1 (s + 5),rJ) | T, ] 
-E [ F(s, p 2 (s) , p 2 (s + S) , p 2 , q 2 (s) , q 2 (s + S) , q 2 , r 2 (s) , r 2 (s + S) , r 2 ) | F s } ) } ds 



<2Ef e Xs [\p 1 (s) -p 2 (s)\ 



s+8 



xC [Ip 1 ^) - p 2 (s)\ + \p 1 (s + 5) - p 2 (s + 5)\+ J Wiu) -p 2 (u)\ du 



s+S 



+ \q 1 (s)-q 2 (s)\ + \q 1 (s + 8)-q 2 (s + S)\ + J \q x (u) - q 2 (u)\ du 



s+S 



+ jr^s) - r 2 (s)\ n + jr^s + 5) - r 2 (s + S)\ n + J jr 1 (u) - r 2 (u) \ H du 



ds 



By the above inequalities for (p, q, r) and the fact that 2ab < h eb 2 , we 

e 

have that 



E 



e xt \p 1 (t)-p 2 (t)\ 



+ E 



J e Xs \q 1 (s)-q 2 (s)\ 2 ds 



+E 



J e Xs J ^(s, z) - r 2 (s,z)\ 2 dsv(dz) 



t R 

+(2 + e~ xs ) eE 



J e x s |p 1 (s) ~p 2 (s)\ ds 
t 

Je Xs \p 1 (s)-p 2 (s)\ 2 ds 



+ (2 + e~ x s )eEje Xs \q\s) - q 2 (s)\ 2 + \r\s, z) - r 2 (s, z)^ 



ds 
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Taking e such that (2 + e~ Ad )e = 



E 
+E 



e xt \p 1 {t)~p 2 (t)\ 



+ E 



J e Xs \q 1 (s)-q 2 (s)\ 2 ds 



j e x s J |r 1 (s, z) — r 2 (s, z)\ 2 dsv{dz) 



,ZC 2 



<(—-X+- 2 )E 



j e Xs jp^s) - p 2 (s)| 2 ds 



1 



-E 



f\qi(s)-q 2 (s)\ 2 ds 

t 

OO 

/ jr^s, z) - r 2 (s,z)\ z n ds 



We get 



E 



,A t |„1 



P Ht)-P 2 (t) 



e \q (s) - q (s)\ ds 



/ e A s / |r 1 (s, z) - r 2 (s, z)\ dsv(dz) 



J e x s |p 1 (s) -p 2 (s)| ds 



3C 2 1 

Using the fact that A > 1 — , we obtain for all t G [0, oo), 

6 2 



E 



e xt \p 1 {t)-p 2 {t)\ 



0, 



which proves that p x {t) and p 2 (i) are indistinguishable. ■ 

Acknowledgment 5.2 JVe thank Salah-E.A. Mohammed for fruitful conver- 
sations. 
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